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L Abstract 

Performance Seeking Control attempts to find the 
operating condition that will generate optimal perfor- 
mance and control the plant at that operating condition. In 
this paper a nonlinear multivariable Adaptive Perfor- 
mance Seeking Control (APSC) methodology will be 
developed and it will be demonstrated on a nonlinear 
system. The APSC is comprised of the Positive Grandient 
Control (PGC) and the Fuzzy Model Reference Learning 
Control (FMRLC). The PGC computes the positive gradi- 
ents of the desired performance function with respect to 
the control inputs in order to drive the plant set points to 
the operating point that will produce optimal perfor- 
mance. The PGC approach will be derived in this paper. 
The feedback control of the plant is performed by the 
FMRLC. For the FMRLC, the conventional fuzzy model 
reference learning control methodology is utilized, with 
guidelines generated here for the effective tuning of the 
FMRLC controller. 

2. Introduction 

Control techniques utilized to drive the plant to pro- 
duce optimal performance are found in an area that is 
called Performance Seeking Control (PSC)/ 1 ' 7 ) Conven- 
tional PSC control approaches compute the optimal per- 
formance off-line utilizing some control algorithm like 
linear programming , a gradient, or some neural net method. 
The optimal operating point or trajectory is then passed to 
an on-line feedback controller for the control of the 
process. The APSC structure proposed in this paper, 
Fig. l,isin essence a PSC approach, but because the com- 
putations are entirely performed on-line, in a closed loop 
control fashion, it is more appropriately classified here as 
an adaptive control approach. 
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In this paper it will be shown that an on-line APSC has 
been realized through the computation of the positive 
gradients, (for adesiredperformance function) with respect 
to the plant control inputs. These gradients are used to 
drive the plant set points in a closed loop fashion. When 
this optimal operating condition is reached the gradients 
of the performance function with respect to the control 
inputs will all be zero, and the control will stop driving the 
process set points any further. In this work, the combined 
effect of PGC and FMRLC with its ability to perform 
nonlinear control, with fast on-line learning of the control 
law, will be exploited. 

During the past several years, fuzzy control has 
emerged as one of the most active and promising control 
areas, especially because of the ability of fuzzy control in 
controlling highly nonlinear, time variant, and ill-defined 
systems. The works of Mamdani and his colleagues on 
fuzzy control^ 12-15 ) was motivated by Zadeh’s work on 
the theory of fuzzy sets/ 16 ' 19 ) and its application to 
linguistics and systems analysis. The work of Procyk and 
Mamdami on the linguistic self-organizing controller^ 20 ) 
as well as refinements to this algorithm made by others, 
was later modified and extended by Layne to what it is 
called FMRLC/ 21 ) The FMRLC structure. Fig. 2, has 
learning capabilities and differs conceptually from adap- 
tive control primarily by its ability to memorize learned 
experiences. The FMRLC algorithm will be utilized here 
for nonlinear, multivariable feedback control, and some 
guidelines will be generated for the effective tuning of the 
FMRLC controller. In this paper the PGC and FMRLC 
controllers will be combined to form the new on-line 
APSC structure shown in Fig. 1. 

3.0 Adaptive Performance Seeking Control 

The APSC structure proposed in this paper is shown 
in Fig. 1. The feedback control of the state variables is 
performed by the FMRLC in a nonlinear multivariable 
control structure shown in more detail in Fig. 2. The APSC 
is initialized with a switch in the open position, and the set 
points, r t , are controlled remotely. When the switch is 
closed, the control of the set points is automatically 
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Figure 1. — Adaptive performance seeking control structure. 
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updated by the APSC. It should be noted that in this 
operating mode the remote portion of the set points can 
still be updated as in a trim control fashion. The APSC 
remains continuously active even when the maximum 
performance has been reached by preventing the gradient 
from falling exactly to zero. This is shown by the limit 
bands built around zero in Fig. 1. The limit band of the 
control derivatives is chosen larger than the correspond- 
ing limit band of the performance function in order to 
prevent large gradient excursions for very small changes 
in the control inputs. In addition, the limit bands around 
zero will prevent the maximum point due to a certain 
gradient direction from being approached in the limit 
sense. This will provide for the establishment of a new 
gradient direction towards the maximum performance 
point. The dashed line blocks in Fig. 1 are derivative 
approximations. 

3.1 Plant Description 

To facilitate the development of this control method- 
ology the following nonlinear system is presented and 
analyzed: 

ij = -2x x x 2 + 3x 2 + «i 

( 1 ) 

X 2 = X\ “ *2 + «2 


The process itself (i.e. with zero control input) is 
determined to be stable by using the Liapunov Direct 
method, with the Liapunov function: V(x l9 x 2 ) = axf m 
+ bx 2 n . With the choices ofm=l,n=l,a=l,b = 2, which 
simplifies V(x), V(x x jc 2 ) = x\ + 2x| which is positive 
definite. V(x) = W(x(t)) T g(x(t)), and with no control 
input, V (x v x 2 ) = -4x\ + 6x x x 2 which is negative semi- 
definite as long as the inequality 2x 2 ^ 3x/s satisfied. 

Section 3 will cover the development of the APSC 
control structure shown in Fig. 1, with the derivation of 
the PGC and the discussion of the FMRLC control 
approach. In section 4 the simulation results for the APSC 
structure will be presented. Section 5 will cover the 
conclusion. 

3.2 Positive Gradient Control 

Based on the process in Eq. (1) and the performance 
function in Eq. (2): let / be a function of two variables 
x 1 and x 2 . Also, for simplicity let x x be a function of an 
independent variable u x and x 2 be a function of an 
independent variable u 2 . It is desired to find the point 
(xf, x 2 *), where/assumes its maximum value, flxf, x 2 *). 
A necessary condition for (xf, x 2 *) to be a point where/ 
has a relative maximum is that the differential off vanish 
at (xf , x/), that is, 


where (x x ,x 2 ), (u v u 2 ) are the states of the system and the 
control inputs respectively. 


/ * *\ df dx i / * *\ 


dx x diii 


Au\ 


The process in (1) is chosen to be nonlinear, stable, 
with strong cross coupling of the control inputs to the 
controlled variables. Further, a performance function is 
selected to demonstrate this control structure which is a 
function of the states and with the properties of continuity, 
convexity, and quadratic, where: 

f(x 1 > x 2 ) = -(*1 - C 1 f - ( x 2 - c 2 f + c 3 ( 2 ) 


+ 


<y dx 2 

dx 2 du 2 



A 



Au= 0 


( 3 ) 


This function describes an elliptic paraboloid, with a 
maximum easily determined by inspection to be equal to 
cl at (xj,x 2 ) = (c lt c 2 ), where c v c 2 , c 3 are constants. The 
performance function in (2) could have also been extracted 
from a corresponding performance index as the argument 
inside the integral of the performance index, except for the 
fact that the desire here is to maximize this function 
instead of minimizing it. In addition atypical performance 
index could contain a penalty function for control expen- 
diture, but this portion of the control development will not 
be carried out in this paper. 


df/du is the gradient of/ with repect to m. Since u x and u 2 
are independent, the components of A u are independently 
arbitrary and (3) implies 

d f( X *) = 0. (4) 

du 

In Fig. 1, instead of using the gradient dfidu i to 
control the setpoints of the states, the derivative expression 
fliii is utilized. To use the gradient expression would 
required the knowledge of an analytic function 
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/(jc 1 ,jc 2 ,...,jc 0 ,« 1 ,w 2 ,...,w m ) defined at every point 
(x x The substitution of the derivative expression 

in place of the gradient of / necessitates an analysis to 
compare their behavior, in order to determine whether the 
derivative expression will produce the desirable 
results. For simplicity, lets assume that/is a function of 
two variables, u x and u 2 , which are in turn a function of 
t. The derivative expression of 


direction in order to maximize the function/, then the state 
x x is commanded to move in the right direction. If the state 
Xjis already at or past the point that would maximize the 
function/, then a positive second term in Eq. (7) would 
move the state in the wrong direction. However, moving 
the state in the wrong direction relative to maximizing the 
function/ will cause the numerator sign of the second term 
in Eq. (7) to become negative, thereby forcing the state to 
move back in the right direction. 


f_ 

u x 


" df dii] | df du 2 \ ldu\ 
K du\ dt du2 dt jj dt 


can be expanded in the limit sense as: 


( 5 ) 


/ 

«1 

/(«! , M 2 + ) ~ /(“ 1 >“ 2 ) Au 2 

Au 2 At 


/(«! +Ali ] ,U 2 )- /(«i , « 2 ) t 

Ait] At 



( 6 ) 


After some cancellation of terms Eq. (6) reduces to: 


/ /(«! + Ami,M 2 ) — /(“!» “2) | 

Mj Au x 

f(u u U2+Au 2 )-f(u l ,u 2 ) 

A Ul j 

Similarly the gradient of / with respect to u x can be 
expressed as: 


9/ = /(hi+Amj, M 2 ) -/(«!, M 2 ) g) 

3 u x A u x 

Inspection of Eqs. (7) and (8) shows that the two 
differ by the second term in Eq. (7) which is absent in 
Eq. (8). Now lets examine how the control is expected to 
behave with the substitution of (7) for (8). The gradient of 
the performance function with respect to the control input 
u x in Eq. (8) represents the desirable direction of control 
adjustment of the state x 1 set point which maximizes the 
function/ When the first term of Eq. (7) is much greater 
than its second term, Eq. (7) reduces to Eq. (8). In the 
worst case, when the second term in Eq. (7) is much larger 
than its first term, the state x x is adjusted primarily due to 
the change of/ relative to the control input u 2 instead of 
u x . If the second term in Eq. (7) is positive greater than the 
first term, and the state x x still needs to move in a positive 


The gradient vector is normal to the elevation contours 
and at each point it has the direction of maximum increase 
of the function/ The vector representing the derivative 
approximation to the gradient will not be exactly normal 
to the elevation contours of / nevertheless, the derivative 
vector establishes a certain positive ascending direction 
towards maximizing /. This approximate PGC method- 
ology can also be thought as providing a series of excitations 
to the control system, with each excitation forcing the 
states closer to the optimum performance point. 

Based on the above, controlling the process in the 
positive gradient direction will essentially follow an 
ascending path on the performance surface described by 
the performance function in (2), much like a hill climbing 
problem. When a positive direction path is established the 
control will follow this trajectory to the point where 
climbing stops. At this point a new positive gradient 
direction is established and climbing towards the maxi- 
mum point resumes. This process is repeated until finally 
the maximum performance point is reached. When this 
maximum performance point is reached the gradient dfidu 
in (4) will be zero and the control will cease to update the 
process setpoint, thereby allowing the process to settle on 
this operating point. With the limit bands built around the 
zero points shown in Fig. 1, the control will be making 
small excursions around this maximum performance point 
in order to continuously hunt for this maximum. In this 
proposed control structure the plant model is not needed 
for the actual control of the process. However, for the 
fuzzy controller, a rather simple fuzzy model of the plant 
is constructed. This will be discussed in the next section. 

3.3 Fuzzy Model Reference Learning Control 

Fuzzy control theory will not be covered in depth in 
this paper. For more detail discussions in these areas see 
Refs. 1 1 to 22. The FMRLC structure (Ref. 20), shown in 
Fig. 2, employs an inverse fuzzy model of the process and 
modifies the knowledge base through the knowledge base 
modifier mechanism in order for the process output y{kt) 
to match the reference model output ym(kt). In this section 
the basic design procedure of the FMRLC for the process 
in Eq. (1) will be discussed. 
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For the MIMO system discussed in this paper two 
decoupled FMRLC controllers are constructed. A coupled 
FMRLC controller could be utilized instead, however, the 
dimensions of the knowledge bases would have increased 
equivalent to the number of the inputs to the fuzzy 
controller. In addition to the basic FMRLC structure 
shown in Fig. 2, a pole at zero frequency was placed at the 
output of each decoupled controller. This is needed for 
zero steady state error. Each decoupled FMRLC controller 
contains 6 adjustable gains. Therefore, some discussion in 
this section will be devoted to establishing some guidelines 
for the effective tuning of the control gains. Typical inputs 
to the fuzzy controller are the error e(kT) and the error 
derivative c(kT ), but other types of inputs can be chosen 
such as integration of the error. The membership functions 
for all the inputs to the fuzzy controllers and the inverse 
models have been chosen with triangular shape, 
normalized, and uniformly distributed in each Universe of 
Discourse, as shown in Fig. 3. In Fig. 3, E 7 signifies a 
membership function or linguistic value associated with a 
specific input to the fuzzy controller, where |ll gives the 
certainty that an element of that particular input may be 
classified heuristically as E J . Figure 4 shows the rule base 
constructed for the inverse fuzzy models. From this rule 
base it can be deduced that the Consequent membership 
functions corresponding to the inverse model output 
variable yj(kT) have similar distribution to the membership 


functions shown in Fig. 3. The knowledge base (rule 
base) contains the centers of the membership functions 
which are triangular shaped for this problem, with a base 
width of 0.4 as seen in Fig. 3. One of the important 
consideration in the construction of the inverse knowledge 
base is that the inverse fuzzy model exhibits the proper 
directionality associated with the controlled process. The 
knowledge base associated with the fuzzy controllers 
initially contains all zeros, which reflects no knowledge 
of how to control the process. This knowledge base is 
updated automatically as the FMRLC controller learns 
how to control the process. 

The selection of the FMRLC gains is an important 
step in the design process, as the ability of the controller 
to track the reference model will heavily depend on the 
particular choices of the gains. The gains g e and g y are 
chosen so that the ranges of these inputs are mapped to 
a normalized universe of discourse in the range of [-1 ,1]. 
For instance an appropriate choice for the value of the 
gain g e would be 1 irange (e(kt)). A good choice for the 
value of the gain g c is found to be approximately equal to 
10 f (range (e(kt))/T ), which is equal to 10/(max change 
(i r(kt))/T ), where rikt) is the set point and T is the sampling 
time. The smaller the choice for the values of the gains 
g e and g c , the more the control action is concentrated 
towards the center region of the rule base, resulting in 



-1 .0 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 +1 .0 


Figure 3. — Membership functions. 


1 -5-4-3-2-10+1 +2 +3 +4 +5 

-5 -1.0 -1.0 -1.0 -1.0 -1.0 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 

_4 -i.o -1.0 -1.0 -1.0 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 

-3 -1.0 -1.0 -1.0 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 

-2 -1.0 -1.0 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 

_1 -i.o -1.0 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 

yj 0 -1.0 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 +1.0 

+1 -0.8 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 +1.0 +1.0 

+2 -0.6 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 +1.0 +1.0 +1.0 

+3 -0.4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 +1.0 +1.0 +1.0 +1.0 

+4 -0.2 0.0 +0.2 +0.4 +0.6 +0.8 +1.0 +1.0 +1.0 +1.0 +1.0 

+5 0.0 + 0.2 + 0.4 + 0.6 + 0.8 + 1.0 + 1.0 + 1.0 + 1.0 + 1.0 + 1.0 

Figure 4. — Inverse fuzzy model rule base. 
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better control tracking at the expense of an increased 
control rate of the control variable u(kT). The gain, g y , 
effects the damping of the process response: If it’s too 
small the response will be oscillatory, if it’s too large, the 
process will be unable to keep up with the reference 
model. A good choice for the value of the gain, g y , is 
found to lie somewhere in the range of [l/(4co w ), l/(2co n )], 
where, co n , is the natural frequency of the process. The 
output gains , g u and gj , are chosen so that the corresponding 
Normalized Universe of Discourse maps to the range of 
the output variables of the fuzzy controller. For instance, 
both g u and gy are selected to be equal to the range of the 
control input variable, u(kT). This choice for the output 
gains allows both u(kT) and yfltT) to take on values as 
large as the largest control input. 

The selection of the reference model shown in Fig. 2, 
represents the desired performance of the FMRLC feedback 
control system. The reference model is selected here to 
have a natural frequency, co w , equal to the process natural 
frequency, co n , with a relatively low step value for the open 
loop response. With the process being nonlinear, its 
response time can strongly depend on the magnitude of the 
control input. Therefore, it may not be desirable to select 
a reference model significantly faster than the process 
response time relative to a low control input value, or else 
we may be asking for relatively large control rates. A first 
order model for the selection of the reference model has 
been found to be adequate. 


g rm - 


0>m 

S + to m 


4.0 APSC Simulation 


( 9 ) 


The APSC simulation consists of two parts. The first 
part is the simulation of the FMRLC and the correspond- 
ing tuning of its gains as discussed in section 3.3. The 
second part is the simulation of the overall APSC control- 
ler shown in Fig. 1 with the combined FMRLC and PGC 
control structure. 

4.1 FMRLC Control Simulation 

Based on the discussion in section 3.3, the control 
parameters for the two decoupled FMRLC controllers 
have been selected with the following values: 

8e\ 8y e 1 8 cl <?;y c l 8ul 8fl ®ml 

8e 1 8y e 2 8c 2 8y c 2 8 U 2 8/2 ^m2_ 

0.25 0.25 0.1 0.125 10 10 2" 

0.25 0.25 0.1 0.125 10 10 2 


The defuzzification approach used in this simulation 
is the so called “Center of Gravity.” Figure 5 shows the 
response of the decoupled FMRLC controller with simul- 
taneous step set point changes. This response shows the 
tracking capabilities of the FMRLC. The set point track- 
ing response was used to tune the controller as was 
discussed in section 3.3. The knowledge base of the fuzzy 
controller started with all zero entries, reflecting that 
initially there was no knowledge of how to control the 
system. The learning rate is quite fast as is evident from the 
responses of the states and control inputs in Fig. 5. The 
resulting knowledge base of the decoupled controller 
corresponding to the state x 2 (that was learned from the 
simulation in Fig. 5) is shown in Fig. 6. The zero elements 
associated with this knowledge base is an indication that 
the controller, for this particular simulation, has not had 
the opportunity to venture into these areas of its knowl- 
edge space. 




Figure 5.— FMRLC close loop step response. 
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Figure 6.— Automatically generated rule base for state X 2 - 


4.2 APSC Control Simulation 

The objective of this control simulation is to drive the 
states in (1) to the operating point that will maximize the 
performance function in (2) starting from some arbitrary 
initial conditions (x x (0), x 2 (0), u x (0), w 2 (0)). The resulting 
state trajectory will not be optimal since this approach 
employs an adaptive control structure and no classical 
optimization techniques like linear programming, or steep- 
est descent gradients are used here. 

Additional control parameter values used in this 
simulation as depicted in Fig. 1 are: K- = 0.5, x = 0.25. 
The switch shown in Fig. 1 is closed at t = 1.0 sec. 
Before the switch closes the set points are preset to the 
values (r v r 2 ) - (1.0, 1.0) to start the simulation. After the 
switch closes the setpoints are updated automatically. The 
control moves the states towards the positive gradient of 
the performance function with respect to the control inputs 
in order to find the operating point that will maximize the 
performance function. For this simulation the constants of 
the performance function in (2) have been set to ( c lf c 2 ,c 3 ) 
= (2.5, 2.0, 453 ) which causes the optimal performance 
value to be/* = 5.0 at (xf, x$) = (2.5, 2.0). 

Figure 7 shows the performance function as it moves 
to its maximum obtained value, and Fig. 8 shows the two 
states as they transition to the operating point correspond- 
ing to the performance function in Fig. 7. It is evident from 
Fig. 8 that this transition to the optimal operating point 
occurs as a series of responses that more closely resemble 
first order type system responses. Figure 9 shows the input 
control response for the same simulation. Figure 10 shows 
the combined state trajectory on the three-dimensional 
performance surface of Eq. (2). Figure 1 1 shows a contour 
map depicting elevation contours of the performance 
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Figure 9.— Control trajectories. 


Adaptive performance seeking control 



Figure 10.— Combined state trajectory assent. 


function in Eq. (2), and the combined state trajectory to 
the highest elevation point. It is evident from Fig. 1 1 that 
the ascent to the top of the performance surface follows a 
series of ascenting paths, where a certain path ends and a 
new path is established when climbing ceases in that 
particular direction. This type of assent is the direct result 
of following a positive gradient path, where the rate of 
assent is proportional to the magnitude of the gradients at 
each time instant. Figure 12 shows different state trajecto- 
ries starting from various initial conditions and all con- 
verging to the highest elevation point on the performance 
elevation contour map. 

For these simulations, the knowledge bases of the two 
decoupled FMRLC controllers were also initialized with 
zeros, but the resulting knowledge bases from the simula- 
tion in Fig. 6 could have been used as the starting point. 


Adaptive performance seeking control 



Figure 1 1 .—Combined state trajectory assent 
through elevation contours. 


Adaptive performance seeking control 



State xl 

Figure 1 2.— Assent of multiple state trajectories. 


5.0 Conclusion 

In this paper a nonlinear process was used to help 
develop an Adaptive Performance Seeking Control meth- 
odology. This methodology utilize the Fuzzy Model Ref- 
erence Learning Control method and an approximate 
Positive Gradient Control approach which was developed 
in this paper. The simulation results presented in this paper 
showed that the FMRLC, with the discussed tuning guide- 
lines, provides for an effective way to control nonlinear 
and tightly coupled processes. The results also show that 
the approximate Positive Gradient Controller within the 
closed loop Adaptive Performance Seeking Control struc- 
ture effectively drives the process to operate at the point 
that generates maximum performance. 
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Since a mathematical model of the plant was not 
used in the control structure described in this paper, and 
since relative to the plant only its natural frequency 
information was utilized to tune the controller, it would be 
expected that this control structure would be adaptive to 
changes in the plant dynamics to the extent that there are 
no large variations to the plant natural frequency. In 
addition, since the APSC controller continuously hunts 
for the operating condition that generates maximum per- 
formance, it may become feasible to perform engine 
control without the need of extensive testing to derive 
engine control schedules. 

For future work it would be important to study 
adaptiveness of this control methodology to plant model 
variations, stability, convergence, and robustness in more 
detail. Further, experimental validation of this method 
would be needed, with processes that exhibit more com- 
plex system dynamics. 
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